feat: eliminate LEFT JOINs whose right side is unused#23566
Open
simonvandel wants to merge 2 commits into
Open
feat: eliminate LEFT JOINs whose right side is unused#23566simonvandel wants to merge 2 commits into
simonvandel wants to merge 2 commits into
Conversation
Extend the `EliminateJoin` rule to remove a LEFT JOIN entirely (replacing it with its left input) when: 1. none of the right side's columns are referenced above the join, and 2. the join cannot multiply left rows: either the right side is provably unique on the equi-join keys (PRIMARY KEY / UNIQUE constraints, GROUP BY, DISTINCT), or the join's ancestors are duplicate-insensitive. A LEFT JOIN preserves every left row whether or not it matches, so under these conditions the join has no observable effect. A join filter does not block the rewrite: for a left join it only decides whether a row is matched or null-padded, and either way the row is emitted. Such joins commonly appear in generated SQL and in queries over views that join in lookup tables the query does not read. This reuses the live-column tracking, duplicate-insensitivity context, and join-key uniqueness analysis that `EliminateJoin` already uses to rewrite inner joins to semi joins. Includes sqllogictest coverage for the eliminated and preserved cases, including result correctness for unmatched and duplicated rows.
github-actions
Bot
added
optimizer
neilconway
reviewed
Jul 16, 2026
neilconway
left a comment
neilconway
left a comment
Contributor
There was a problem hiding this comment.
Thanks for working on this -- lgtm overall!
Nit: The PR description will be the git commit message when this is squash-merged, so it's confusing to talk about "first commit" vs "second commit" there.
The existing analysis has both unit tests and SLTs; can you take a look at whether any of the existing unit tests should be extended to cover outer joins?
| // be replaced by its left input. A join filter cannot prevent this: it | ||
| // only decides whether a left row is matched or null-padded, and either | ||
| // way the row is emitted. | ||
| if join.join_type == JoinType::Left |
Contributor
There was a problem hiding this comment.
What about supporting RIGHT JOINs as well?
Contributor
Author
There was a problem hiding this comment.
Sure, would you be okay with this being a follow-up task? Then we could land the left-join support faster
Comment on lines
+418
to
+423
| && (duplicate_insensitive | ||
| || side_unique_on_join( | ||
| join.right.schema(), | ||
| join.on.iter().map(|(_, right)| right), | ||
| join.null_equality, | ||
| )) |
Contributor
There was a problem hiding this comment.
Consider refactoring this into a helper, so we can use it here and also in rewritten_join_type?
| reset datafusion.execution.batch_size; | ||
|
|
||
| ########## | ||
| # Eliminate unused LEFT JOINs (`EliminateJoin` rule) |
Contributor
There was a problem hiding this comment.
Can we add test cases for LIMIT and LEFT JOIN LATERAL?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue does this PR close?
Rationale for this change
Join elimination is useful in e.g. generated queries or views, where joined columns can end up not being used.
An explanation of when the optimization is valid, can be seen in the docs update for
EliminateJoin.What changes are included in this PR?
Builds upon the analysis that #22652 introduced for the
EliminateJoinoptimization pass. We use it to detect when the right-side of a left-join can be removed.First commit adds test cases.
Second commit adds the optimization change, and shows how it affects the added tests.
Are these changes tested?
Yes, SLT additions that test the feature end-to-end.
Are there any user-facing changes?
Yes, faster queries!